Crate llmvm_protocol
source ·Expand description
This module contains protocol types and utilities used for communicating with llmvm core & backends.
Uses multilink to communicate with local & remote processes.
Modules§
- Protocol error types.
- JSON-RPC types and methods.
async fn(Request) -> Result<Response, Error>- Miscellaneous utility functions.
Structs§
- Request for language model generation.
- Response for language model generation.
- Parameters used for generation via core service. Can be saved in a preset and/or directly provided within the
GenerationRequest. - Request for text generation via core service.
- Response for text generation via core service.
- A prompt or generated message from a thread.
- A parsed model id data structure.
- A “one size fits all” error type for the protocol. Contains a boxed error, and the error type.
- Metadata for a thread.
Enums§
- The actor who presented the message.
- A response container returned by a multilink service.
Constants§
- Default request timeout.
Traits§
- The backend service which the core uses to generate text. Implements a low-level interface for interacting with language models.
- A configuration data structure that provides an example for generating new TOML configuration files. The example should include customizable fields with comments explaining their purpose.
- The core service which frontends use to interact with language models. Manages & uses threads, presets, prompt templates and backend connections to create & send backend requests.
Type Aliases§
- A boxed dynamic type for multilink services. The service must return a result with a
ServiceResponseorServiceError. - A stream of multiple response results returned by the service.
- A boxed error type that may be returned by service calls.
- A future that returns a result with a generic response and
ServiceError. This is returned by service calls.